CHAPTER 11 Comparing Average Values between Groups 155

Interpreting the output of an ANOVA

We describe the R output here, but output from other statistical packages will have

similar information. The output begins with the variance table (or simply the

ANOVA table). You can tell it is a table because it looks like it has a column with no

heading followed by columns with the following headings: Df (for df), Sum Sq (for

the sum of squares), Mean Sq (mean square), F value (value of F statistic), and

Pr(>F) (p value for the F test). You may recall that in order for an ANOVA test to be

statistically significant at α = 0.05, the p value on the F must be < 0.05. It is easy

to identify that F = 12.59 on the output because it is labeled F value. But the p value

on the F is labeled Pr(>F), and that’s not very obvious. As you saw before, the p

value is in scientific notation, but resolves to 0.00000353, which is < 0.05, so it is

statistically significant.

If you use R for this, you will notice that at the bottom of the output it says Signif.

codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1. This is R explaining its coding sys-

tem for p values. It means that if a p value in output is followed by three asterisks,

this is a code for < 0.001. Two asterisks is a code for p < 0.01, and one asterisk

indicates p < 0.05. A period indicates p < 0.1, and no notation indicates the p value

is greater than or equal to 0.1 — meaning by most standards, it is not statistically

significant at all. Other statistical packages often use similar coding to make it

easy for analysts to pick out statistically significant p values in the output.

Several statistical packages that do ANOVAs offer one or more post-hoc tests as

optional output, so programmers tend to request output for both ANOVAs and

post-hoc tests, even before they know whether the ANOVA is statistically signifi-

cant or not, which can be confusing. ANOVA output from other software can

include a lot of extra information, such as a table of the mean, variance, standard

deviation, and count of the observations in each group. It may also include a test

for homogeneity of variances, which tests whether all groups have nearly the same

SDs. In R, the ANOVA output is very lean, and you have to request information like

this in separate commands.

Executing and interpreting post-hoc t tests

In the previous example, the ANOVA was statistically significant, so it qualifies for

post-hoc pairwise t tests. Now that we are at this step, we need to select which

adjustment to use. We already have an idea of what would happen if we used the

Bonferroni adjustment. We’d have to run t tests like we did before, only this time

we’d have to use the three-level MARITAL variable and run three t tests: One with

M and NM, a second with M and OTH, and a third with M and OTH. For each p value

we got, we would have to compare it to the adjusted Bonferroni α of 0.016 instead

of 0.05. By evaluating each p value, you can determine which pairs of groups are

statistically significantly different using the Bonferroni adjustment.